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Research  Activities 
1  Singular  value  decompositions 

The  singular  value  decomposition  (SVD)  has  proved  to  be  a  useful  tool  in  signal 
processing.  Many  of  the  most  powerful  methods  such  as  ESPRIT  which  is  used  for 
directional  of  arrival  estimation  in  sensor  array  processing  use  the  SVD  as  a  basic 
building  tool.  We  have  gone  on  to  generalize  this  decomposition  for  two  or  more 
matrices  and  these  generalizations  are  very  useful  in  many  applications  arising  in 
signal  processing  and  systems  theory. 

As  an  example,  the  restricted  singular  value  decomposition  (RSVD)  is  the  factoriza¬ 
tion  of  a  given  matrix,  relative  to  two  other  given  matrices.  It  can  be  interpreted  as 
the  ordinary  singular  value  decomposition  with  different,  inner  products  in  row  and 
column  spaces.  Its  properties  and  structure  have  been  investigated  in  detail  as  well  as 
its  connection  to  generalized  eigenvalue  problems,  canonical  correlation  analysis  and 
other  generalizations  of  the  singular  value  decomposition. 

Applications  include  the  analysis  of  the  extended  shorted  operator,  unitarily  invariant 
norm  minimization  with  rank  constraints,  rank  minimization  in  matrix  balls,  the 
analysis  and  solution  of  linear  matrix  equations,  rank  minimization  of  a  partitioned 
matrix  and  the  connection  with  generalized  Schur  complements,  constrained  linear 
and  total  linear  least  squares  problems,  with  mixed  exact  and  noisy  data,  including  a 
generalized  Gauss-Markov  estimation  scheme. 

A  constructive  proof  of  the  RSVD  based  upon  the  ordinary  and  the  product  singular 
value  decomposition  has  been  derived. 

(Joint  work  with  Bart  De  Moor) 


2  Chebyshev,  Krylov  and  Lanczos 

We  consider  functions  that  are  orthogonal  with  respect  to  a  given  symmetric  inner- 
product.  A  new  derivation  of  the  fundamental  relationship  between  orthogonal  func¬ 
tions  and  their  representation  in  non- orthogonal  bases  is  given. 

Given  a  basis  of  linearly  independent  polynomials,  the  modified  Chebyshev  algorithm 
constructs  the  set  of  orthogonal  polynomials  corresponding  to  the  inner-product.  We 
present  a  characterization  of  the  algorithm  in  terms  of  transformations  between  poly¬ 
nomial  bases.  This  leads  to  generalized  Krylov  sequences  and  demonstrates  the  equiva¬ 
lence  of  the  modified  Chebyshev  algorithm  and  the  Lanczos  algorithm  for  determining 
eigenvalues  of  linear  operators. 

Block  generalized  Krylov  sequences  (of  which  the  block  Lanczos  algorithm  is  a  special 
case)  are  shown  to  produce  matrix  polynomials  whose  linearization  is  the  matrix  used 
to  generate  the  Krylov  sequence. 

This  theoretical  work  leads  to  a  better  understanding  of  the  important  Lanczos  al¬ 
gorithm  and  its  variants.  Immediate  applications  include  determining  optimal  pa¬ 
rameters  and  error  bounds  for  certain  iterative  methods  for  solving  systems  of  linear 
equations. 

(Thesis  of  Mark  Kent) 
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3  Quadratic  problems  with  constraints 

In  many  situations  in  signal  processing,  one  needs  to  minimize  a  quadratic  form  sub¬ 
ject  to  a  quadratic  constraint  or  to  linear  constraints  which  are  inhomogeneous.  The 
quadratic  constraint  problem  requires  the  solution  of  a  lagrange  multiplier  that  satis¬ 
fies  a  quadratic  eigenvalue  problem.  The  difficulty  in  solving  the  lagrange  multiplier  is 
that  the  eigenvalues  of  the  original  matrix  must  be  known  precisely.  We  have  recently 
developed  an  elegant  algorithm  which  yields  upper  and  lower  bounds  on  the  lagrange 
multiplier.  The  key  idea  is  to  approximate  the  secular  equation  by  an  integral  and 
then  bound  the  integral  using  the  ideas  of  Gauss-Radau  integration. 

The  Lanczos  algorithm  is  used  for  determining  the  orthogonal  polynomials  which 
are  required  when  computing  the  quadrature  rule.  The  details  of  this  approach  will 
soon  appear  in  a  report,  “A  Constrained  Least  Squares  Problem”,  which  is  being  co¬ 
authored  by  Golub  and  Von  Matt. 


4  Orthogonal  Polynomials 

Orthogonal  polynomials  occur  in  many  contexts  in  applied  mathematics,  but  are  es¬ 
pecially  relevant  to  data  fitting.  They  have  the  property  of  stabilizing  the  delicate 
problem  of  fitting  polynomials  to  data  in  least  squares  problems.  In  many  situations, 
the  data  arrives  in  a  sequential  manner,  and  this  requires  the  updating  of  the  coeffi¬ 
cients  of  the  three  term  recursions  used  in  generating  the  polynomials. 

In  some  cases,  one  has  a  sliding  window  so  that  a  new  point  is  added  as  one  point  of 
data  is  deleted.  We  have  developed  algorithms  which  allow  for  performing  these  calcu¬ 
lations  in  a  very  satisfactory  manner.  Our  study  of  updating  orthogonal  polynomials 
has  led  to  algorithms  for  handling  data  where  several  processors  are  available.  That 
is,  we  are  able  to  subdivide  a  large  set  of  data  into  subsets  and,  then  after  computing 
the  recursion  relations  on  each  subset,  we  are  able  to  combine  the  coefficients  in  an 
efficient  manner.  This  leads  to  a  highly  parallel  algorithm  which  is  also  quite  efficient. 
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5  Beam  Forming 

In  various  applications,  it  is  necessary  to  keep  track  of  a  low-rank  approximation  of 
a  covariance  matrix,  R(t),  slowly  varying  with  time.  It  is  convenient  to  track  the  left 
singular  vectors  associated  with  the  largest  singular  values  of  the  triangular  factor, 
L(t),  of  the  Cholesky  factorization.  These  algorithms  are  referred  to  as  “square  root”. 
The  drawback  to  those  applications  is  the  volume  of  the  computational  burden  of 
the  Eigenvalue  Decomposition  or  the  Singular  Value  Decomposition  (SVD).  Various 
numerical  methods  carrying  out  this  task  have  been  performed,  and  we  show  why  this 
point  is  in  fact  questionable  in  numerous  situation.-  and  should  be  revised.  Indeed, 
the  complexity  per  eigenpair  is  generally  a  quadratic  function  of  the  problem  size,  but 
there  exist  faster  algorithms  whose  complexity  is  linear.  Finally  in  order  to  maie  a 
choice  among  the  large  and  fuzzy  set  of  available  techniques,  we  have  made  compar¬ 
isons  based  on  computer  simulations  in  the  relevant  signal  processing  context. 

A  report,  “Tracking  a  Few  Singular  Values  and  Vectors  in  Signal  Processing”,  by 
Pierre  Comon  and  Gene  Golub  has  recently  been  completed  and  submitted  for  pub¬ 
lication. 


6  Least  squares  with  a  quadratic  constraint. 

We  consider  the  following  problem:  Compute  a  vector  x  such  that  ||  Ax  -  b||2  =  min, 
subject  to  the  constraint  ||x||2  =  a.  A  new  approach  to  this  problem  based  on 
Gauss  quadrature  has  been  derived.  The  method  is  especially  well  suited  where  the 
dimensions  of  A  are  large  and  the  matrix  is  sparse. 

The  heart  of  the  new  method  consists  in  the  computation  of  a  partial  bidiagonalization 
of  the  matrix  A,  which  is  often  all  one  can  do  in  these  settings. 

(Joint  work  with  Urs  Von  Matt). 


7  Recursive  condition  estimation. 

Estimates  for  the  condition  number  of  a  matrix  are  required  in  many  application 
areas  of  scientific  computing,  including:  optimization,  least  squares  computations, 
eigenanalysis,  and  general  nonlinear  problems  solved  by  linearization  techniques.  Our 
purpose  is  to  develop  some  adaptive  condition  estimators,  based  on  the  Lanczos  algo¬ 
rithm,  and  test  them  on  recursive  least  squares  (RLS)  computations  arising  in  control 
and  signal  processing.  RLS  algorithms  are  known  to  suffer  from  numerical  instability 
problems  under  finite  word-length  conditions. 

We  have  provided  an  adaptive  Lanczos  schemes  for  estimating  the  smallest  and  largest 
singular  values  of  R,  a min(R)  and  <rmax(R),  respectively,  for  each  recursive  update 
or  downdate  step  for  R  (or  R~l  in  the  covariance  method).  The  computations  are 
adaptive  in  the  sense  that  estimates  at  time  t  are  used  to  obtain  estimates  at  time 
t+1. 

(Joint  Work  with  W.  Ferng  and  R.  Plemmons) 
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8  The  Lanczos  algorithm  and  controlability 

We  derive  a  non-symmetric  Lanczos  algorithm  that  does  not  require  strict  bi-orthogonality 
among  the  generated  vectors.  We  show  how  the  vectors  generated  are  algebraically 
related  to  ’’Controllable  Space”  and  ’’Observable  Space”  foT  a  related  linear  dynamical 
system. 

The  Lanczos  Algorithm  was  originally  proposed  by  Lanczos  as  a  method  for  the  com¬ 
putation  of  eigenvalues  of  symmetric  and  nonsymmetric  matrices.  The  idea  was  to 
reduce  a  general  matrix  to  tridiagonal  form,  from  which  the  eigenvalues  could  be  easily 
determined.  The  nonsymmetric  Lanczos  Algorithm  has  received  much  less  attention 
than  the  symmetric  case.  Besides  some  numerical  stability  problems,  the  method 
suffers  from  the  possibility  of  a  breakdown  from  which  the  only  way  to  “recover”  was 
to  restart  the  whole  process  from  the  beginning  with  different  starting  vectors  .  This 
problem  was  not  solved  until  comparatively  recently. 

The  Lanczos  Algorithm  is  an  example  of  a  method  that  generates  bases  for  Krylov 
subspaces  starting  with  a  given  vector.  In  previous  paper  ,  we  have  examined  how 
another  closely  related  method,  the  Arnoldi  Algorithm,  may  be  used  to  compute 
the  controllable  space  for  a  linear  time-invariant  dynamical  system.  The  Arnoldi 
Algorithm  can  be  thought  of  as  a  ’’one-sided”  method,  which  generates  one  sequence 
of  vectors  that  span  the  controllable  space.  We  have  extended  this  idea  to  the  use 
of  a  two-sided  method,  the  non-symmetric  Lanczos  Algorithm,  which  generates  two 
sequences  of  vectors  spanning  the  left  and  right  Krylov  spaces  corresponding  to  the 
controllable  and  the  observabt(le  spaces.  We  have  demonstrated  how  the  vectors  are 
generated  in  such  a  way  that  we  obtain  bases  not  only  for  the  left  and  right  Krylov 
spaces,  but  also  for  the  intersections  of  these  spaces  and  the  complementary  spaces. 
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